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DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

1. A request for continued examination under 37 CFR 1.114, including tlie fee set fortli in 
37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible 
for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has 
been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 
CFR 1.1 14. Applicant's submission filed on 09 April 2009 has been entered. 

Response to Amendment 

2. This Office Action is responsive to applicant's request for continued examination. As 
such, applicant has amended claims 1 and 16. Claims 1 to 20 remain pending. A complete 
response to applicant's remarks is stated here below. 

Ciaim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained ttiougti ttie invention is not identically disclosed or described as set 
fortti In section 102 of ttils title, If ttie differences between ttie subject matter sougtit to be patented and 
ttie prior art are sucti ttiat ttie subject matter as a wtiole would tiave been obvious at ttie time ttie 
Invention was made to a person tiaving ordinary skill In ttie art to wtilcti said subject matter pertains. 
Patentability stiall not be negatived by ttie manner In wtilcti ttie Invention was made. 

4. Claims 1-4 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Herley (US 20040264806 A1) in combination with Benn et al. (US 5734740 A), Pollard (US 
2004/0028271 Al) and Nicolas (US 20020172417 Al). 



Regarding Claim 1: (currently amended) Herley teaches a method for a processor to color 
match a first image and a second image, wherein a first region of the first image and a second 
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region of tine second image overlap ("The composite signal generator described herein provides 
a system and method for automatically generating a composite image of a scene from a set of 
two or more images of the scene. The resulting composite image has reduced or eliminated 

areas of occlusion relative to the images in the set. The composite is generated by first aligning 
or registering the images." at paragraph [0059]) the method comprising: 

Benn also teaches color matching a first image and a second image, wherein a first region of 

the first image and a second region of the second image overlap (Refer to Figure 5, wherein 

numeral S6 states "Divide digital image into sub-images"); 

generating a first histogram of the first region (Refer to Figure 5, numeral Sv5); 

generating a second histogram of the second region (Refer to Figure 5, numeral S8); 

Pollard teaches determining at least one parameter of an optoelectronic conversion function 
(OECF) ("The processing itself may be performed according to the steps outlined in the flow- 
chart of FIG. 7. These include a pre-processing stage 92, which may typically include correction 
of the OECF (opto-electronic conversion function) of the sensor and white-balancing to 
compensate for variations in illumination." at paragraph [0108]); 

color matching the second image to the first image by applying the OECF with the at least one 
parameter to the second image ("Following the colour correction and de-mosaicing stage 94 
described above, a subsequent post-processing stage 96 may include exposure correction 
(which can also be accomplished at the preprocessing stage) and transformation to a standard 
colour space such as RGB (as described in I EC 61966-2-1)." at paragraph [0108]). 
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Nicoias teaches determining corresponding pixel values from the first and the second 
histograms (Refer to paragraph [0033]; "Only pixels of the first set of pixels are used to generate 
the first and the second histogram."); 

Altematively, Nicolas teaches color matching the second image to the first image by applying a 
function with the at least one parameter to the second image ("The receiving means 302 can get 

notified or has capabilities to extract information from the video signal about the type of scene 
that has been imaged in order to set the appropriate range of color values of the image display 
apparatus 300. E.g. in the case of a football match the predetermined range of colors should 
match the colors of grass." at paragraph [0068]). Although Nicolas does not expressly teach an 
OECF, it is obvious to one of ordinary skill in the art that an OECF function was known in the 
prior art at the time of the invention. This claimed limitation would have been obvious because 
the skilled artisan could have easily substituted the OECF function for any algorithm or routine 
as taught by Nicolas and this would have yielded the same predictable results to the skilled 
artisan to obtain the color matching results of the first and second images. 

All the claimed elements were known in the prior art and one skilled in the art could have 
combined the elements as taught by Herley in combination with Benn, Pollard, and Nicolas by 
known methods with no change in their respective functions, and the combination of those 
teachings would have yielded predictable results to one of ordinary skill in the art at the time of 
the invention. 

Herley Benn, Pollard, and Nicolas are combinable because they are in the same field of color 
image processing, specifically, color correction. 
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The suggestion/motivation for combining the teachings of Herley Benn, Poliard, and Nicolas 
would have been to classify pixels via histogram processing and based on that classification, 
color matching, determining whether the pixels belong to texture or to flat regions by comparing 
2 histograms, one taking into account all the pixels with the color in the predetermined range of 
color values, the second one counting only these same pixels if their luminance value differs 
more than a given threshold form the luminance of their neighbor., at abstract Nicolas. 

Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings 
of Herley Benn, Pollard, and Nicolas to obtain the specified claimed elements of Claim 1 . 

Regarding Claim 2 (original): Herley teaches removing a percentage of the overlapping pixels 
with the greatest difference in brightness ("In general, histogram equalization can be used to 
stretch or compress the brightness of the pixels making up an image based on the overall 
distribution of pixel brightness levels in the image." at paragraph [0079]) 

Herley does not expressly recite removing a percentage of overlapping pixels with the greatest 

difference in brightness; however, the skilled artisan could have yielded the same predictable 
results with no change in the respective functions since this particular technique of histogram 
equalization is recognized as a part of the ordinary capabilities of one skilled in the art. 

Regarding Claim 3: (original): Benn teaches said generating a first histogram comprises 
recording in a first plurality of pixel value bins a first plurality of numbers of pixels that have 
respective pixel values in the first region (Refer to Figure 4, numeral S5) and said generating a 
second histogram comprises recording in a second plurality of pixel value bins a second 
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plurality of numbers of pixels that have the respective pixel values in the second region (Refer to 
Figure 4, numeral S8). 

Regarding Claim 4: (previously presented): Herley teaches determining corresponding pixel 
values from the first and the second histograms comprises generating a lookup table (LUT) 
storing a third plurality of numbers of pixels and their corresponding pixel values ("From this 

count, a cumuiative distribution function is computed and normalized to a maximum value 
corresponding to the number of pixel brightness levels employed. The cumuiative distribution 
function is then used as a lookup table to map from the original pixel brightness levels to final 
levels." at paragraph [0080]). 

Regarding Ciaim 16 (currently amended): Herley teaches a method for a processor to color 
match a first image and a second image, wherein a first region of the first image and a second 
region of the second image overlap ("The composite signal generator described herein provides 
a system and method for automatically generating a composite image of a scene from a set of 
two or more images of the scene. The resulting composite image has reduced or eliminated 
areas of occlusion relative to the images in the set. The composite is generated by first aligning 
or registering the images." at paragraph [0059]) the method comprising: 

removing a percentage of overtapping pixels with the greatest difference in brightness ("in 
general, histogram equalization can be used to stretch or compress the brightness of the pixels 
making up an image based on the overall distribution of pixel brightness levels in the image." at 
paragraph [0079]); 
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Benn teaches generating a first tiistogram of tlie first region and a second liistogram of tlie 
second region after said removing (Refer to Figure 5, numeral S5); 
Nicolas teaches histogram matching the first and the second histograms to determine 
corresponding pixel values from the first and the second histograms (Refer to paragraph [0033]; 
"Only pixels of the first set of pixels are used to generate the first and the second histogram."). 

Pollard teaches minimizing a color matching error between the corresponding pixel values, 
wherein the color matching error is generated from an optoelectronic conversion function 
(OECF) ("The processing itself may be performed according to the steps outlined in the flow- 
chart of FIG. 7. These include a pre-processing stage 92, which may typically include correction 
of the OECF (opto-electronic conversion function) of the sensor and white-balancing to 
compensate for variations in illumination." at paragraph [0108]); 

color matching the second image to the first image by applying the OECF to the second image 
("Following the colour correction and de-mosaicing stage 94 described above, a subsequent 

post-processing stage 96 may include exposure correction (which can also be accomplished at 
the preprocessing stage) and transformation to a standard colour space such as RGB (as 
described in lEC 61966-2-1)." at paragraph [0108]). 

Alternatively, Nicolas teaches color matching the second image to the first image by applying a 
function with the at least one parameter to the second image ("The receiving means 302 can get 
notified or has capabilities to extract information from the video signal about the type of scene 
that has been imaged in order to set the appropriate range of color values of the image display 
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apparatus 300. E.g. in the case of a footbail matcli tlie predetermined range of colors stiouid 
match the colors of grass." at paragraph [0068]). Although Nicolas does not expressly teach an 
OECF, it is obvious to one of ordinary skill in the art that an OECF function was known in the 
prior art at the time of the invention. This claimed limitation would have been obvious because 
the skilled artisan could have easily substituted the OECF function for any algorithm or routine 
as taught by Nicolas and this would have yielded the same predictable results to the skilled 
artisan to obtain the color matching results of the first and second images. 

Herley Benn, Pollard, and Nicolas are combinable because they are in the same field of color 
image processing, specifically, color correction. 

All the claimed elements were known in the prior art and one skilled in the art could have 
combined the elements as taught by Herley in combination with Benn, Pollard, and Nicolas by 
known methods with no change in their respective functions, and the combination of those 
teachings would have yielded predictable results to one of ordinary skill in the art at the time of 
the invention. 

The suggestion/motivation for combining the teachings of Herley Benn, Poliard, and Nicolas 
would have been to classify pixels via histogram processing and based on that classification, 
color matching, determining whether the pixels belong to texture or to flat regions by comparing 
2 histograms, one taking into account all the pixels with the color in the predetermined range of 
color values, the second one counting only these same pixels if their luminance value differs 
more than a given threshold form the luminance of their neighbor., at abstract Nicolas. 
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Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings 
of Herley Benn, Pollard, and Nicolas to obtain the specified claimed elements of Claim 16. 

5. Claims 11 and 12 are rejected under 35 U.S. C. 103(a) as being unpatentable over 
Herley (US 20040264806 A1) in combination with Benn et al. (US 5734740 A), Pollard (US 
2004/0028271 Al) and Nicolas (US 20020172417 Al) and further in view of Hasler "Modeling 
the Opto-Electronic Conversion Function (OECF) For Application in the Stitching of Panoramic 
Images, hereinafter referred to as Hasler- (MOASPI). 

Regarding Claim 11: Herley in combination with Benn, Pollard, and Nicolas teach all the 

claimed elements as rejected above. 

The combination of Herley, Benn, Pollard, and Nicolas does not expressly teach the OECF is 

S{.x) -Jt-i- — iirciiiff\ -- , 

defined as: " wherein 8 () is the OECF, x is a pixel 

value normalized to (0, 1), and a(epsilon)(-1,1) is a first color matching parameter 

Hasler- (MOASPI) teaches the OECF is defined 

Six}-^^—&rct&f^ .L..-i-- , 

as: wherein S() is the OECF, x is a pixel value 

normalized to (0,1), and a(epsilon)(-1,1) is a first color matching parameter (Refer to Equation 1, 
right column, section 3, "The OECF Model"). 
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Herley, Benn, Pollard, Nicolas and Hasier are combinable because they are in the same field of 
image segmenting in panoramic images and opto-electronic conversion functionality. 

At the time the invention was made, the skilled artisan could have defined the OECF function as 
recited above. All the claimed elements were known in the prior art at the time of the invention. 

The suggestion/motivation to combine the teachings of Herley, Benn, Poilard, and Nicolas with 
Hasier would have been the OECF with these definitive elements can "deliver an optimal result 
in a least square error sense." (Hasier, abstract). 

Therefore, at the time of the invention, it would have been obvious to the skilled artisan to 
combine the teachings of Herley, Benn, Pollard, and Nicolas with Hasier to obtain the specified 
claimed elements of Claim 1 1 . 

Regarding Claim 12 (original): Hasier- (MOASPI) teaches determining at least one parameter 
of an OECF comprises minimizing a color matching error defined as: 



wherein e is the color matching error, x.sub.1 and x.sub.2 are corresponding pixel values in the 

first and the second regions, R.sub.1 and R.sub.2 are the first and the second regions, S( ) is 
the OECF, S( ).sup.-1 is the inverse OECF, and (tau) is a second color matching parameter 
(Refer to Equation 2d, left column, paragraph 2, section 4 "The Error Metric"). 
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6. Claim 13 is rejected under 35 U.S.C. 103(a) as being unpatentable over Herley (US 
20040264806 A1) in combination with Benn et al. (US 5734740 A), Pollard (US 2004/0028271 
A1), Nicolas (US 20020172417 Al), Hasler "Modeling the Opto-Electronic Conversion Function 
(OECF) For Application in the Stitching of Panoramic Images, hereinafter referred to as Hasler- 
(MOASPI) and further in view of Press, "The Art of Scientific Computing, 10.1 Golden Search in 
One Dimension'-hereinafter referred to as Press. 

Regarding Claim 13: (original): Herley in combination with Benn, Pollard, Nicolas and Hasler 

teach all the claimed elements as rejected above. 

Herley in combination with Benn, Pollard, Nicolas and Hasler does not specifically teach 
minimizing a color matching error comprises performing a golden section search of the color 
matching error. 

Press teaches minimizing a color matching error comprises performing a golden section search 
of the color matching error (For example, refer to equations (10.1.6) and (10.1.7) in reference to 
the golden mean or golden section search examples). 

Herley, Benn, Pollard, Nicolas, Hasler and Press are combinable because they are in the same 
field of image segmenting in panoramic images and golden section searching. 

At the time the invention was made, it would have been obvious to one of ordinary skill in the art 
to minimizing a color matching error comprises performing a golden section search of the color 
matching error. 
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The suggestion/motivation would have been "the golden section search guarantees that each 
new function evaluation will bracket minimum to an interval a precise number times the size of 
the preceding interval." (Hasler, page 399-400, last paragraph, final sentence). 

Therefore, at the time of the invention, it would have been obvious to the skilled artisan to 
combine Herley, Benn, Pollard, Nicolas, Hasler and Press to obtain the specified claimed 
elements of Claim 13. 

Allowable Subject Matter 

7. Claims 5-10, 14, 15 and 17-20 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the limitations of 
the base claim and any intervening claims. 

Response to Arguments 

8. Applicant's arguments with respect to claims 1 -4, 1 1 -1 3 and 1 6 as argued at pages 9-1 6 
have been considered but are moot in view of the new ground(s) of rejection. See newly 
rejected claims above. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Mia M. Thomas whose telephone number is (571)270-1583. The 
examiner can normally be reached on Monday-Thursday 8am-5pm. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh M. Mehta can be reached on 571-272-7453. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private 
PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you 
would like assistance from a USPTO Customer Service Representative or access to the 
automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Andrew W Johns/ Mia M Thomas 

Primary Examiner, Art Unit 2624 Examiner 

Art Unit 2624 



